Tool for optimizing system performance and methods relating to same

ABSTRACT

One or more embodiments of the invention enable improved communication with a database comprising multiple clients utilizing multiple large data objects concurrently. For example when a client system interacts with a server with respect to a data object that is over a threshold size, the system may utilizing a communication methodology that minimizes system resource usage such as CPU utilization and network utilization. In one embodiment of the invention when a client request for an object falls within the relevant size threshold, one or more embodiments of the invention segment the object into smaller size chunks. Hence the server is not required to assemble all data associated with a request at once, but is instead able to immediately start transmitting smaller segments of data. Allowing for the transmission of smaller data chunks prevents the server from allocating large blocks of memory to one object and although the server may be required to handle more memory allocations, each allocation is smaller in size and can therefore be processed much faster. The determination of the chunk size is dependent on inherent system resources such as the amount of server memory, and the available bandwidth of the network. In addition, the determination of chunk size is dependent on environmental factors such as the time of day, the day of the week, the number of users, the number of predicted users for a given time and day based on historical logging, and the current and predicted network utilization for a given time and day. One or more embodiments of the invention obtain the chunk size and optionally obtain a chunk transfer size from a server that may alter these quantities dynamically in order to minimize resource utilization.

BACKGROUND OF THE INVENTION

U.S. patent application entitled “METHOD AND APPARATUS FOR MANAGING DATAOBJECTS IN A MULTI-USER ENVIRONMENT”, filed Dec. 30, 2004 to the sameinventor is hereby incorporated herein by reference.

1. Field of the Invention

Embodiments of the invention described herein pertain to the field ofcomputer software. More particularly, but not by way of limitation, oneor more embodiments of the invention determine a blob chunk thresholdand optional chunk transfer size to enable efficient communication witha database comprising binary large object (BLOB) data.

2. Description of the Related Art

Database systems generally provide the ability to permanently store andretrieve various types of data including characters, numbers, dates,objects, images, video and other types of data such as binary largeobject data. It has become increasingly more common as hard drivecapacities and network bandwidth have continually increased for databaseusers to store and retrieve large data objects such as multimediaincluding images, videos and audio. Storing and retrieving large dataobjects can inhibit the efficiency of a DBMS system and reduce overallsystem performance. This is because most database systems are notdesigned to handle widespread access to large data objects and hencethere is a significant burden placed on the system when access to theselarge data objects becomes a regular part of day to day operations. Thisreduced performance during storage and retrieval of large objects iscaused by a number of different factors.

When a large data object is retrieved by a client the database reads theentire object with one read operation. This retrieval operation in turncauses the database to allocate a segment of memory on the databaseserver in which to read the entire data object being retrieved. On asmall scale such allocations can reduce performance, but do notnecessarily cause a substantial drop in overall system performance.However, when such requests become more common and many users across theentire system request large objects, systems are required to handle manythreads asking for similar functions thus causing a significantreduction in system performance. If for instance, 30 users initiaterequests to retrieve different PDF objects where each object isapproximately 100 mb in size, then the server allocates approximately 3Gb of memory. In many cases the occurrence of such an allocationrequirement will impact system performance.

The impact of retrieving and storing large data objects on memory occurswhen a database performs commands such as insert, update and select forexample. Microsoft SQL Server, for instance, typically allocates 4 timesthe amount of memory of the object to be inserted. So in cases where a50 MB object is to be inserted the server allocates approximately 200 MBof memory to the insert task. Another performance related problem thatoccurs when large data objects are transmitted between a client andserver is that the transmission of such objects causes an increase inthe number of network collisions which in turn places a noticeableburden on the network and reduces overall system efficiency.

To alleviate the burdens placed on a system when utilizing large blocksof data a technique known as “blob chunking” is used to read smallerblocks of data from a BLOB field until the entire BLOB field is read.Blob chunking may also be used to write a series of smaller blocks ofdata to a BLOB field until the entire BLOB field is written. To datethere has been no way to intelligently determine the size of blocks tobreak a read or a write of a BLOB field into as current attempts at BLOBchunking merely attempt to allow a system to operate without returningan “out of memory” error for example. In addition, the threshold atwhich the blob chunking is utilized may not be the best size fortransferring data to minimizing network utilization. The blob chunk sizeand transfer chunk size may differ as the memory of a database systemand the network throughput of a system comprising the database havetheir own specific characteristics and utilizations that vary in timegenerally independently.

Because of the limitations described above there is a need for a systemthat can determine and optimize the blob chunk size threshold andoptionally determine a transfer chunk size to allow greater efficiencyin situations where large data objects are accessed and transferredbetween a server and a client.

SUMMARY OF THE INVENTION

One or more embodiments of the invention determine a blob chunkthreshold and optional chunk transfer size to enable efficientcommunication with a database serving multiple large data objects tomultiple clients. For example when a client system interacts with aserver with respect to a data object that is over a threshold size, thesystem may employ a communication methodology that minimizes systemresource usage such as CPU and memory utilization by breaking a databaseaccess into more numerous smaller accesses. Optionally, the smalleraccesses may be broken or combined into transfer chunk sizes thatoptimize use of the network.

In one or more embodiment of the invention software elements areemployed to monitor the database system memory utilization and CPUutilization and calculate a blob chunk size threshold. Optionally, anetwork software element is employed to determine the throughput andutilization of the network and calculate a transfer chunk size to usefor network transfer of the data. Both the software element and thenetwork software element may execute once at system startup, on an eventbasis or continuously at configurable intervals and field requests forblob chunk size and transfer chunk size. The software element and thenetwork software element may utilize historical utilization informationin order to predictively alter the blob chunk size and the transferchunk size. When configured to access the calculated blob chunk size andoptional transfer chunk size client machines and servers may communicateusing smaller segments of data. Allowing for the access and transmissionof smaller data chunks prevents the server from allocating large blocksof memory to one object and although the server may be required tohandle more memory allocations, each allocation is smaller in size andcan therefore be processed much faster.

The determination of the chunk size is dependent on inherent systemresources such as the amount of server memory, and the availablebandwidth of the network. In addition, the determination of chunk sizeis dependent on environmental factors such as the time of day, the dayof the week, the number of users, the number of predicted users for agiven time and day based on historical logging, and the current andpredicted network utilization for a given time and day.

Software applications that utilize embodiments of the invention may ormay not have knowledge of the underlying communications that make use ofthe blob chunk size and optional transfer chunk size. For example aftera blob chunk size and optional transfer chunk size are calculated aserver informs the client system it will be sending a multi-part objectand also identifies the size the object will be upon reassembly and thentransmits smaller data blocks back to the requesting client. Softwareapplications may initiate requests to the server for a blob chunk sizeand optional transfer chunk size as well. For example the client machinemay make a request to the server for a given object and obtain a messageinforming the client of the blob chunk size and optional transfer sizeat which time the client may request the blob chunks, each of which maycomprise a different transfer chunk size that varies in time as theclient and server CPU and memory utilization and network utilizationchanges or is predicted to change. The user of the client machine may ormay not be aware that the blob chunk size and optional transfer chunksize exists as created and updated by one or more embodiments of theinvention or that the data is being segmented and streamed for deliverythrough use of the blob chunk size or optional transfer chunk sizequantities. Each block transferred may be of a different size than aprevious or subsequent block as the optimal chunk size for a system maychange over time depending on environmental factors. The change in blocksize may occur during the transmittal of a given data object meaningthat for example the initial chunks sent may be smaller or larger thanthe subsequent chunks sent depending on the number of users or predictednumber of users for a given time or any other factor related to theperformance of the system.

Determining the optimal sizes for blob chunks and transfer chunks asdescribed above is non-trivial since the optimal sizes vary according tothe current utilization of the system and network in which blob chunkingis utilized. Hence one aspect of the invention comprises use ofaveraging memory utilization over time to generate the blob chunk sizeand averaging the network utilization over time to generate a transferchunk size. The window in which to average utilization can be configuredat startup. In addition, historical utilization logs may be utilized inorder to predictively calculate blob chunk size and transfer chunk size.Any mathematical function combining current utilization of memory withhistorical utilization of memory in generating a blob chunk size is inkeeping with the spirit of the invention. In addition, any mathematicalfunction combining current network utilization with historical networkutilization in generating a chunk transfer size is in keeping with thespirit of the invention. Once the blob chunk size is calculated,applications may then use that size in packing, transmitting, unpacking,CRC error checking data to minimize memory utilization on the server.

There are several advantages in utilizing a blob chunk size and optionaltransfer chunk size, as calculated via one or more embodiments of theinvention, with regards to large data objects. For instance, serversmaking use of these quantities can serve a larger number of requests forlarge objects before running out of memory. Because there is no need toallocate server memory for the entire object the system allows forgreater efficiency in instances where the data is read from the serverand passed onto the client. Another benefit occurs in that the number ofnetwork collisions is significantly reduced by transmitting largeobjects to a client in pieces according to the transfer chunk size, ascalculated via one or more embodiments of the invention, rather than inlarger segments. In addition, since the blob chunk size and optionaltransfer chunk size may dynamically change based on performance relatedquantities such as environmental factors, the system remains optimallyresponsive regardless of external events.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a hardware architectural view of an embodiment of theinvention.

FIG. 2 illustrates an architectural view comprising software elementsused in one or more embodiments of the invention.

FIG. 3 illustrates a flow chart showing an embodiment of the method usedin calculating a blob chunk size.

FIG. 4 illustrates a flow chart showing an embodiment of the method usedin dynamically calculating a blob chunk size.

FIG. 5 illustrates a flow chart showing an embodiment of the method usedin predictively calculating a blob chunk size.

FIG. 6 illustrates a flow chart showing an embodiment of the method usedin functionally calculating a blob chunk size.

FIG. 7 illustrates a flow chart showing an embodiment of the method usedin dynamically functionally calculating a blob chunk size.

FIG. 7 illustrates a flow chart showing an embodiment of the method usedin dynamically functionally calculating a transfer chunk size.

DETAILED DESCRIPTION OF THE INVENTION

A tool for optimizing system performance and methods relating to samewill now be described. In the following exemplary description numerousspecific details are set forth in order to provide a more thoroughunderstanding of embodiments of the invention. It will be apparent,however, to an artisan of ordinary skill that the present invention maybe practiced without incorporating all aspects of the specific detailsdescribed herein. In other instances, specific features, quantities, ormeasurements well known to those of ordinary skill in the art have notbeen described in detail so as not to obscure the invention. Readersshould note that although examples of the invention are set forthherein, the claims, and the full scope of any equivalents, are whatdefine the metes and bounds of the invention.

FIG. 1 illustrates a hardware architectural view of a system that maymake use of an embodiment of the invention to determine a blob chunkthreshold and optional chunk transfer size to enable efficientcommunication with a database serving multiple large data objects tomultiple clients. Client 100 and client 101 communicate with server 104over network 102 via router 103 to obtain blob chunk size and transferchunk size, retrieve and store data objects in database 105. For examplewhen a client system such as client 100 and/or 101 interacts with server104 with respect to a data object stored in database 105 that is over athreshold as per the blob chunk size, the system may utilize acommunication methodology that takes into account the blob chunk size soas to minimize system resource usage such as CPU utilization, memoryutilization and/or network utilization. The communication methodology iscapable of transmitting data utilizing the blob chunk size or optionallya transfer chunk size each of which may change over time based on eventsthat may alter the performance of the system.

FIG. 2 illustrates a software architectural view of one or moreembodiments of the invention. Data chunking interface module 202 iscapable of calculating a blob chunking size and optional transfer chunksize on command, as a result of an event or at timed intervals. Thedetermination of the chunk size is dependent on inherent systemresources such as the amount of server memory, and the availablebandwidth of the network. In addition, the determination of chunk sizeis dependent on environmental factors such as the time of day, the dayof the week, the number of users, the number of predicted users for agiven time and day based on historical logging, and the current andpredicted network utilization for a given time and day. In one or moreembodiments of the invention the calculation of the chunk size takesinto account the current state of the users and memory utilized in thesystem and the predicted number of users and memory and networkbandwidth utilized based on historical observations. As the transfer ofdata chunks is spread over time, embodiments of the invention arecapable of changing the chunk size and chunk size threshold dynamicallywhile a transfer is taking place. Note that although a chunk size isused as a threshold and transfer size for data, these two elements maybe of differing size as one skilled in the art will appreciate. Forexample, a chunk size of 100 MB may be used to segment any data objectover 100 MB into chunks of 100 MB, or alternatively, a chunk size of 100MB may be used to segment any data object over 100 MB into 25 MBtransfer chunks. Regardless of the data chunk size or optional transferchunk size, embodiments of the invention are capable of obtaining andutilizing these elements statically at configuration time or dynamicallywhile running.

In one or more embodiments of the invention, data chunking interfacemodule 202 calculates and provides access to the blob chunk size whichis calculated in one or more embodiments according to the current memoryutilization via memory utilization checker 251 and optionally CPUutilization checker 252. Memory utilization checker 251 may alsodetermine the number of users on the system. Database driver 203 isqueried to determine the number of database users that are currentlycoupled with the system. If the network utilization is low then the blobchunk size may be used as the transfer chunk size. Optionally, datachunking module 201 may request a maximum blob chunk size and transferchunk size for example if client 100 has limited memory orcommunications capabilities. Therefore, blob chunk size and transferchunk size may vary per client. If the current network utilization ishigh, then a transfer chunk size may be used for the size of the datachunks transferred between data chunking module 201 and data chunkinginterface module 202. In addition to using the current utilizationvalues of the system, data chunking interface module 202 is capable ofutilizing historical information in log 275. Log 275 may be implementedas one data repository as shown, or multiple independent logs that storeutilization information related to the number of users, number ofdatabase users, memory utilization, CPU utilization and networkutilization with respect to time. Using historical time stampedutilization quantities allows the system to predict for example that theperiod between noon and 1 PM for example comprises fewer database usersand lower network utilization allowing for a higher blob chunk sizeequal to the transfer chunk size. Another example period may comprisealternate historical values, for example Monday at 3 PM which maycomprise heavy usage of the network and medium utilization of the servermemory allowing for a medium blob chunk size and a transfer chunk sizecalculated that differs from the blob chunk size and allows for moresparing use of the network.

Use of the blob chunk size is accomplished via data chunking module 201and data chunking interface module 202. For example, in one embodimentof the invention when a client request from client application 200 foran object (such as a video stored in a database) falls within therelevant size threshold, data chunking module 201 and data chunkinginterface module 202 are capable of transferring the object between themusing smaller size chunks. Hence server 104 is not required to assembleall data associated with a request at once, but is instead able toimmediately start transmitting smaller segments of data to client 100.The chunking can be hidden from the user as when a database package iswritten to implement the chunking without the knowledge of a client, orwith an interface that differs from the standard SQL interface.Regardless of the interface involved, any method of utilizing a chunksize threshold according to the description of the embodiments detailedherein is in keeping with the spirit of the invention. Allowing for thetransmission of smaller data chunks prevents server 104 from allocatinglarge blocks of memory to one object and although the server may berequired to handle more memory allocations, each allocation is smallerin size and can therefore be processed much faster.

When utilizing embodiments of the invention, the latency between thetime of the request and the delivery of data is typically low. Settingthe transfer chunk size to a value smaller than the blob chunk size maydecrease the latency of the first chunk of data arriving at client 100,however the overall time to send a blob may increase since more chunksare sent requiring more handshakes. In one embodiment of the inventionwhen server 104 transmits smaller data blocks back to requesting client100, server 104 informs client 100 that it will be sending a multi-partobject and also identifies the size the object will be upon reassembly.Hence client 100 is made aware the request is being met by a stream ofmulti-part data to assemble rather than by a single object even thoughthis may be transparent to the user of client 100. Each blocktransferred may be of a different size than a previous or subsequentblock as the optimal chunk size for a system may change over timedepending on environmental factors. The change in block size may occurduring the transmittal of a given data object meaning that for examplethe initial chunks sent may be smaller or larger than the subsequentchunks sent depending on the number of users or predicted number ofusers for a given time.

Software applications that utilize embodiments of the invention may ormay not have knowledge of the underlying communications that make use ofthe blob chunk size and optional transfer chunk size. For example aftera blob chunk size and optional transfer chunk size are calculated bydata chunking interface module 202 running on server 104 informs theclient system (either client 100 or 101) that server 104 will be sendinga multi-part object and also identifies the size the object will be uponreassembly and then transmits smaller data blocks back to requestingclient 100. Software applications such as client application 200 mayinitiate requests to server 104 for a blob chunk size and optionaltransfer chunk size as well for example before sending information toserver 104 for storage in database 105. For example client machine 101may make a request to server 104 for a given object and obtain a messagecomprising the blob chunk size and optional transfer size at which timeclient 101 may request the blob chunks, each of which may comprise adifferent transfer chunk size that varies in time as the client andserver CPU and memory utilization and network utilization changes or ispredicted to change. The user of the client machine may or may not beaware that the blob chunk size and optional transfer chunk size existsas created and updated by one or more embodiments of the invention orthat the data is being segmented and streamed for delivery through useof the blob chunk size or optional transfer chunk size quantities. Eachblock transferred may be of a different size than a previous orsubsequent block as the optimal chunk size for a system may change overtime depending on environmental factors. The change in block size mayoccur during the transmittal of a given data object meaning that forexample the initial chunks sent may be smaller or larger than thesubsequent chunks sent depending on the number of users or predictednumber of users for a given time or any other factor related to theperformance of the system.

FIG. 3 illustrates a flow chart showing an embodiment of the method usedin calculating a blob chunk size. This embodiment is exemplary in thatany environmental quantity may be utilized in determining the blob chunksize. Processing starts at 300 and if there is at least one databaseuser using the system then one embodiment of the invention obtains atime window at 302 over which a memory utilization is obtained at 303 asa number of Megabytes for example. The number of database users usingthe system is obtained at 304 or optionally stored as a result of thecheck at 301. The amount of memory available per database user iscalculated by a division at 305 and the blob chunk size is set to theresult of the division at 306. Optionally, the result of the divisionmay be further divided by a configured constant in order to reservememory to account for users that may begin using the database systemwith BLOBs. Although this is a first order result, it shows that theblob chunk size can be calculated using the current number of databaseusers and the current amount of free memory in the system. If there areno current database users then the maximum number of database userspossible as configured by the database manager is obtained at 307 andthe current amount of free memory in the system is divided by themaximum number of database users at 308 in order to set the blob chunksize at 309. The calculated blob chunk size is returned at 310.

FIG. 4 illustrates a flow chart showing an embodiment of the method usedin dynamically calculating a blob chunk size. This embodiment isexemplary in that any environmental quantity may be utilized indetermining the blob chunk size. Processing starts at 300 and if thereis at least one database user using the system then one embodiment ofthe invention obtains a time window at 302 over which a memoryutilization is obtained at 303 as a number of Megabytes for example. Thenumber of database users using the system is obtained at 304 oroptionally stored as a result of the check at 301. The amount of memoryavailable per database user is calculated by a division at 305 and theblob chunk size is set to the result of the division at 306. Optionally,the result of the division may be further divided by a configuredconstant in order to reserve memory to account for users that may beginusing the database system with BLOBs. Although this is a first orderresult, it shows that the blob chunk size can be calculated using thecurrent number of database users and the current amount of free memoryin the system. If there are no current database users then the maximumnumber of database users possible as configured by the database manageris obtained at 307 and the current amount of free memory in the systemis divided by the maximum number of database users at 308 in order toset the blob chunk size at 309. The calculated blob chunk size isreturned at 310 and after a configurable time period the processingbegins again at 301. This continual processing allows for dynamicupdates of the blob chunk size as the number of users and the amount offree memory varies over time. Any transfers of data that are ongoing mayoptionally update to use the new blob chunk size for transferring dataduring the transfer or continue to use the previously set blob chunksize until the total transfer is complete. Any other function combiningthe amount of memory utilized by a current number or maximum number ofusers is in keeping with the spirit of the invention.

FIG. 5 illustrates a flow chart showing an embodiment of the method usedin predictively calculating a blob chunk size. This embodiment isexemplary in that any environmental quantity may be utilized indetermining the blob chunk size. Processing starts at 500. Oneembodiment of the invention obtains a time window at 501 over which ahistorical memory utilization is obtained from log 275 at 502 as anumber of Megabytes for example. The historical memory utilization maycomprise a calendar based function that may determine if the lastperiod, for example on a weekly basis, was a holiday. For example if thecurrent time of day is 2 PM on a Friday, and if the previous Friday wasa holiday, then the time period 2 weeks before the previous period maybe utilized in obtaining historical memory utilization data.Alternatively, if the current day is a holiday and the previous weeklyperiod did not comprise a holiday on the same day of the week, then thecalendar function may be utilized in order to find the previous timeperiod where a holiday occurred during the week and that historicalmemory utilization quantity may be obtained for the amount of the windowdesired as set in 501. The number of database users using the system isobtained at 503 or optionally the number of users using the systemcorresponding to the historical memory utilization period may beutilized. Alternatively, the maximum number of users may be utilized inthe calculation at 504. The amount of memory available per database useris calculated by a division at 504 and the blob chunk size is set to theresult of the division at 505. The result yields a predictive blob chunksize that corresponds to the amount of predicted traffic that thedatabase system is likely to observe for the given day of the week andtime of day, (holidays notwithstanding). Optionally, the result of thedivision may be further divided by a configured constant in order toreserve memory to account for users that may begin using the databasesystem with BLOBs.

FIG. 6 illustrates a flow chart showing an embodiment of the method usedin functionally calculating a blob chunk size. This embodiment isexemplary in that any environmental quantity may be utilized indetermining the blob chunk size. Processing starts at 500. Oneembodiment of the invention obtains a time window at 501 over which ahistorical memory utilization is obtained from log 275 at 502 as anumber of Megabytes for example. The historical memory utilization maycomprise a calendar based function that may determine if the lastperiod, for example on a weekly basis, was a holiday. For example if thecurrent time of day is 2 PM on a Friday, and if the previous Friday wasa holiday, then the time period 2 weeks before the previous period maybe utilized in obtaining historical memory utilization data.Alternatively, if the current day is a holiday and the previous weeklyperiod did not comprise a holiday on the same day of the week, then thecalendar function may be utilized in order to find the previous timeperiod where a holiday occurred during the week and that historicalmemory utilization quantity may be obtained for the amount of the windowdesired as set in 501. The number of database users using the system isobtained at 503 or optionally the number of users using the systemcorresponding to the historical memory utilization period may beutilized. Alternatively, the maximum number of users may be utilized inthe calculation at 504. The amount of memory available per database useris calculated by a division at 504 and the blob chunk size is set to theresult of the division at 505. Concurrently or serially, the maximumnumber of database users possible as configured by the database manageris obtained at 307 and the current amount of free memory in the systemis divided by the maximum number of database users at 308 in order toset the blob chunk size at 309. The resulting blob chunk size result setat 505 is averaged with the resulting blob chunk size set at 309 inorder to produce a functionally calculated blob chunk size at 600. Oneskilled in the art will recognize that any function may be used in placeof a simple average and may be weighted in terms of a probability thatthe predictive calculation of blob chunk size will be more or lessaccurate than the current available chunk size as calculated by 308.Optionally, the result of the functional calculation of 600 may befurther divided by a configured constant in order to reserve memory toaccount for users that may begin using the database system with BLOBsand this constant may be derived by any method such as for example viaan end of month flag that is set by a calendar function as a predictionof for example sales or marketing personnel attempting to close outtheir month ending projects which make use of BLOBs.

FIG. 7 illustrates a flow chart showing an embodiment of the method usedin dynamically functionally calculating a blob chunk size. Thisembodiment is exemplary in that any environmental quantity may beutilized in determining the blob chunk size. Processing starts at 500.One embodiment of the invention obtains a time window at 501 over whicha historical memory utilization is obtained from log 275 at 502 as anumber of Megabytes for example. The historical memory utilization maycomprise a calendar based function that may determine if the lastperiod, for example on a weekly basis, was a holiday. For example if thecurrent time of day is 2 PM on a Friday, and if the previous Friday wasa holiday, then the time period 2 weeks before the previous period maybe utilized in obtaining historical memory utilization data.Alternatively, if the current day is a holiday and the previous weeklyperiod did not comprise a holiday on the same day of the week, then thecalendar function may be utilized in order to find the previous timeperiod where a holiday occurred during the week and that historicalmemory utilization quantity may be obtained for the amount of the windowdesired as set in 501. The number of database users using the system isobtained at 503 or optionally the number of users using the systemcorresponding to the historical memory utilization period may beutilized. Alternatively, the maximum number of users may be utilized inthe calculation at 504. The amount of memory available per database useris calculated by a division at 504 and the blob chunk size is set to theresult of the division at 505. Concurrently or serially, the maximumnumber of database users possible as configured by the database manageris obtained at 307 and the current amount of free memory in the systemis divided by the maximum number of database users at 308 in order toset the blob chunk size at 309. The resulting blob chunk size result setat 505 is averaged with the resulting blob chunk size set at 309 inorder to produce a functionally calculated blob chunk size at 600. Oneskilled in the art will recognize that any function may be used in placeof a simple average and may be weighted in terms of a probability thatthe predictive calculation of blob chunk size will be more or lessaccurate than the current available chunk size as calculated by 308.Optionally, the result of the functional calculation of 600 may befurther divided by a configured constant in order to reserve memory toaccount for users that may begin using the database system with BLOBsand this constant may be derived by any method such as for example viaan end of month flag that is set by a calendar function as a predictionof for example sales or marketing personnel attempting to close outtheir month ending projects which make use of BLOBs. The calculated blobchunk size is returned at 600 and after a waiting for a configurabletime period at 700 the processing begins again at 500. This continualprocessing allows for dynamic updates of the blob chunk size as thenumber of users and the amount of free memory varies over time. Anytransfers of data that are ongoing may optionally update to use the newblob chunk size for transferring data during the transfer or continue touse the previously set blob chunk size until the total transfer iscomplete. Any other function combining the amount of memory utilized andpredicted amount of memory utilized is in keeping with the spirit of theinvention.

FIG. 8 illustrates a flow chart showing an embodiment of the method usedin dynamically functionally calculating a transfer chunk size. Thisembodiment is exemplary in that any environmental quantity may beutilized in determining the transfer chunk size. Processing starts at800. One embodiment of the invention obtains a time window at 801 overwhich a historical network availability is obtained from log 275 at 802as a number of Megabytes per second for example. Log 275 may compriseutilization and the availability is the total network bandwidth minusthe utilization. The historical network availability may comprise acalendar based function that may determine if the last period, forexample on a weekly basis, was a holiday. For example if the currenttime of day is 2 PM on a Friday, and if the previous Friday was aholiday, then the time period 2 weeks before the previous period may beutilized in obtaining historical network availability data.Alternatively, if the current day is a holiday and the previous weeklyperiod did not comprise a holiday on the same day of the week, then thecalendar function may be utilized in order to find the previous timeperiod where a holiday occurred during the week and that historicalnetwork availability quantity may be obtained for the amount of thewindow desired as set in 801. The number of database users using thesystem is obtained at 803 or optionally the number of users using thesystem corresponding to the historical network availability period maybe utilized. Alternatively, the maximum number of users may be utilizedin the calculation at 804. The amount of availability available perdatabase user is calculated by a division at 804 and the transfer chunksize is set to the result of the division at 805. Concurrently orserially, the maximum number of database users possible as configured bythe database manager is obtained at 806 and the current networkavailability is divided by the maximum number of database users at 807in order to set the transfer chunk size at 808. The resulting transferchunk size result set at 805 is averaged with the resulting transferchunk size set at 808 in order to produce a functionally calculatedtransfer chunk size at 809. One skilled in the art will recognize thatany function may be used in place of a simple average and may beweighted in terms of a probability that the predictive calculation oftransfer chunk size will be more or less accurate than the currentavailable transfer chunk size as calculated by 807. Optionally, theresult of the functional calculation of 809 may be clipped to the blobchunk size or further divided by a configured constant in order toreserve network bandwidth to account for users that may begin using thedatabase system with BLOBs and this constant may be derived by anymethod such as for example via an end of month flag that is set by acalendar function as a prediction of for example sales or marketingpersonnel attempting to close out their month ending projects which makeuse of BLOBs. The calculated transfer chunk size is returned at 809 andafter a waiting for a configurable time period at 810 the processingbegins again at 800. This continual processing allows for dynamicupdates of the transfer chunk size as the number of users and the amountof network availability varies over time. Any transfers of data that areongoing may optionally update to use the new transfer chunk size fortransferring data during the transfer or continue to use the previouslyset transfer chunk size until the total transfer is complete. Any otherfunction combining the amount of current network availability with thepredicted network availability in calculating the transfer chunk size isin keeping with the spirit of the invention.

Thus embodiments of the invention directed to a tool for optimizingsystem performance and methods relating to same have been exemplified toone of ordinary skill in the art. The claims, however, and the fullscope of any equivalents are what define the metes and bounds of theinvention.

1. In a computer system, a method for optimizing system performancecomprising: obtaining a time window via computer readable program code;obtaining a utilization over said time window; obtaining a number ofusers; performing a function on said utilization and said number ofusers; setting a chunk size; and, returning said chunk size for use intransferring a BLOB data type from a database to a client system.
 2. Themethod of claim 1 further comprising: waiting for a configurable timeperiod; and, repeating said obtaining said time window, said obtainingsaid utilization, said obtaining said number of users, said performing afunction, said setting a chunk size and said returning said chunk size.3. The method of claim 1 wherein said utilization is selected from agroup consisting of current and historical utilization.
 4. The method ofclaim 3 wherein said utilization is associated with server memory. 5.The method of claim 3 wherein said utilization is associated withnetwork bandwidth.
 6. The method of claim 1 wherein said performing saidfunction is based on an environmental factor.
 7. The method of claim 6wherein said environmental factor is based on a time of day.
 8. Themethod of claim 6 wherein said environmental factor is based on a day ofthe week.
 9. The method of claim 6 wherein said environmental factor isbased on a number of users.
 10. The method of claim 6 wherein saidenvironmental factor is based on a number of predicted users.
 11. Themethod of claim 6 wherein said environmental factor is based on anetwork utilization.
 12. The method of claim 6 wherein saidenvironmental factor is based on a predicted network utilization.
 13. Acomputer program product for optimizing network performance comprising:a computer usable memory medium having computer readable program codeembodied therein wherein said computer readable program code comprises adata chunking interface module that is configured to: obtain a timewindow via computer readable program code; obtain a utilization oversaid time window; obtain a number of users; perform a function on saidutilization and said number of users; set a chunk size; and, return saidchunk size for use in a transfer of a BLOB data type from a database toa client system.
 14. The computer program product of claim 13 whereinsaid computer readable program code is further configured to: wait for aconfigurable time period; and, repeat said obtain said time window, saidobtain said utilization, said obtain said number of users, said performa function, said set a chunk size and said return said chunk size. 15.The computer program product of claim 13 wherein said utilization isselected from a group consisting of current and historical utilization.16. The computer program product of claim 15 wherein said utilization isassociated with server memory.
 17. The computer program product of claim15 wherein said utilization is associated with network bandwidth. 18.The computer program product of claim 13 wherein said perform saidfunction is based on an environmental factor.
 19. The computer programproduct of claim 18 wherein said environmental factor is based on a timeof day.
 20. The computer program product of claim 18 wherein saidenvironmental factor is based on a day of the week.
 21. The computerprogram product of claim 18 wherein said environmental factor is basedon a number of users.
 22. The computer program product of claim 18wherein said environmental factor is based on a number of predictedusers.
 23. The computer program product of claim 18 wherein saidenvironmental factor is based on a network utilization.
 24. The computerprogram product of claim 18 wherein said environmental factor is basedon a predicted network utilization.
 25. In a computer system, anapparatus configured to calculate a chunk size comprising: means forobtaining a time window via computer readable program code; means forobtaining a utilization over said time window; means for obtaining anumber of users; means for performing a function on said utilization andsaid number of users; means for setting a chunk size; and, means forreturning said chunk size for use in transferring a BLOB data type froma database to a client system.
 26. The apparatus of claim 25 furthercomprising: means for waiting for a configurable time period; and, meansfor repeating said means for obtaining said time window, said means forobtaining said utilization, said means for obtaining said number ofusers, said means for performing a function, said means for setting achunk size and said means for returning said chunk size.